Method and device for multimedia data retrieval

ABSTRACT

The invention concerns a device for multimedia data retrieval. The multimedia data is associated with an active component. The device for multimedia retrieval being characterized in that, depending on an external event, some of the active components trigger an action that make the user aware of the multimedia data associated with said active components.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of co-pending U.S. patent applicationSer. No. 11/547,372 filed on Oct. 3, 2006, the entire contents of whichis incorporated herein by reference.

BACKGROUND

1. Field of technology:

The invention concerns a method and a device to retrieve multimediadata.

2. Discussion of related art:

Driven by the digitalization in the consumer video, photo and audiomarket as well as by increasing capacity of re-writable storage devicessuch as hard discs and DVD, the enormous quantity of multimedia itemsand the problem of multimedia asset management is becoming much moreimportant.

One of the challenges of multimedia asset management is to gathersemantic information from the media in order to allow for easy dataaccess. In the professional market, first products propose semanticaccess. For instance, temporal scenes in video are identified andallowing scene by scene video access. As another example, semanticelements in images such a “persons”, “indoor scenes” or “mountains” areidentified. Videos and images can thus be searched and retrieved bysemantic keywords. Web-based search engines for audio files use semanticinformation such as artist name and title to retrieve audio items.

One feature of multimedia item is its creation date and time, called inthe following, creation date. The creation date is an important featureto retrieve multimedia items. For songs it indicates the creation periodand gives hints to the music style. Further, creation date can be closeto the moment where users have first heard the song. For videos,creation date is often close to date of first appearance in movietheatres or emission on television. Thus, the term creation date can beextended. It may be also the date of first presentation in a country bya specific medium or the date of its comeback into public interest.Creation date may mean also the date of reception of a multimedia item.For example, one receives some video clips from a server duringChristmas holidays. Creation date could be the date of reception, sincethis is first time when the user consumes the multimedia item. Creationdate may even be extended to usage tracking information such as accesshistory.

The creation date is notably important for personal content. This typeof content can be photos, video clips, audio clips, video films or othermultimedia items that are produced or at least transformed at home byend users. User studies showed that creation date is one of the mostimportant features for people to classify and retrieve their multimediaasset. Users often organize their multimedia items by creation date intodirectories. Search and retrieval is often done by remembering creationdate and then retrieving the item by date.

The creation date has affective meaning to users. The creation ofmultimedia items is often linked to events in the personal life ofusers. Photos are often taken during holiday trips, family meetings orweek-ends. Also video clips are often created during holidays or atspecific moments such as weddings or at Christmas. In this way,multimedia items and their creation have a high emotional value tousers. They form a mirror of memory of personal life.

One problem in managing multimedia assets is that the number ofmultimedia items is so large, and its organization is so coarse thatusers of multimedia assets forget, at least temporally, the existence ofspecific items. In case of anniversary of a specific event, users areoften not aware of multimedia items that are linked to this event. Thereason can be that they forgot their existence, or that the creator of aspecific multimedia item was not himself but another person using thesame multimedia asset. Thus it can be another person of the family thatcreated, copied, bought or transformed a multimedia item linked to theevent. Multimedia asset are often so large that even an asset managementsystem would be too busy and don't have enough information to check atevery moment the existence of item that could be interesting to a userin case of anniversary of an event.

SUMMARY

The invention proposes a device for multimedia data retrieval, saidmultimedia data being associated with an active component. According tothe invention, depending on an external event, some of the activecomponents trigger an action that make the user aware of the multimediadata associated with said active components.

Thus the user has not to be aware of the data that exist to retrievethem. The system itself disposes information that the users may not beable to recover easily. The system will provide the user withinformation for which he may not be aware of the existence because hedoes not remember because the information is too old, or because he wasnot the creator of this multimedia data.

The multimedia items stored in the database are active or comprise anactive component which make the retrieval system intelligent in thesense that it is able to retrieve information automatically that couldbe ignored by the user, without any need on the user's side. Thus thedevice can enable the user to access data in an easy way, without anyactive request from the user, and this is enabled by the fact that themultimedia items are linked with active components, such as agents,which are activated upon events independent from the users.

BRIEF DESCRIPTION OF DRAWINGS

Other characteristics and advantages of the invention will appearthrough the description of a non-limiting embodiment of the invention,which will be illustrated, with the help of FIG. 1 which gives anembodiment of a system according to the invention.

DETAILED DESCRIPTION

The following embodiment of the invention concerns a device where theexternal event is related to the date of the current day.

Further embodiments of the invention can be related with other externalevents, based for instance on semantic items.

Thus this embodiment of the invention concerns a method and a devicethat allows multimedia data to remember their existence to the users incase of the anniversary of the creation date. By creation date, one canalso understand the first presentation of the multimedia element, thelast day it was used, the date of reception of the multimedia element,or the date of storage of the multimedia element.

This embodiment is also related to an action which consists indisplaying the multimedia element onto a display. Other actions can beforeseen, such as the sending of the information to users, the musicplaying if the multimedia element comprises a music component.

The multimedia data are stored on a huge capacity storage medium. Inother embodiments, the multimedia data can also be stored on a remotestorage medium and accessed through a network, such as internet forinstance.

The multimedia element is associated with information related to itscreation date as well as with an active component.

The active component is able to process the information on the creationdate of the associated multimedia element as well as the information onthe current date and time. Based on this processing, the activecomponent triggers actions that make the user aware of the existence ofthe multimedia element.

The active component may exploit other information agglomerated with themultimedia element to carry out specific actions such as generation of aremembering message or the display or playing of the multimedia element.

FIG. 1 shows a system including several components realizing anembodiment of the invention.

The system comprises a multimedia asset management system 1 that managesthe multimedia data stored on the storage medium 2.

The storage medium 2 is a hard disk drive but can be also a DVD, or anyother storage means.

The multimedia management system 1 is able to associate with themultimedia data sorted on the storage medium 2, creation dateinformation and an active component. The active components are alsoknown as agents.

The multimedia data stored on the storage medium 2 are able to react toa user interface 3 by sending remembering information to a userinterface or multimedia rendering information to a display 4.

The use interface 3 can receive remembering information and transfersrendering information to a display 4.

The user interface 3 can incorporate a remote control and the display 3can be a digital set-top-box, also known as decoder, associated with adisplay.

The multimedia Asset management system 1 sends a time information 7 tothe storage medium 2 and its associated intelligent processing means.The time information 8 is sent on request of the intelligent processingmeans. The agents associated with the multimedia items react on the timeinformation 8 sent by the multimedia asset management system. When thetime information 7 corresponds to an anniversary of the creation date,the corresponding agent will send the content of the multimedia item tothe user interface 3 or will send to the user interface 3 a rememberinginformation informing the user that the database contains a multimediaitem, for instance a photo representing the first steps of his youngdaughter, on that day some years ago. This is the kind of event a userdoes not remember few years later and that such a system will be able toremember him automatically without any action on the user side. Otherexamples of anniversary are given later on.

The user interface 3 transforms the remembering information intoappropriate information for the display. The display renders, plays ordisplays an interface rendering information 5 as well as a multimediarendering information 6 to the user. The multimedia renderinginformation 6 contains the multimedia element to be remembered to theuser and the interface rendering information 5 contains the way torender this information.

Each multimedia element is composed of several files. Some of the filescontain the multimedia content itself, such as video, still images orsound. Other supplementary files contain supplementary information suchas the creation date. The creation date includes the time and the dateof creation of the multimedia element, or the modification date or thelast access date to the multimedia element.

These supplementary files also contain at least links to all filesbelonging to the same multimedia element, but also other informationsuch as keywords or links to other related data such as other multimediaitems or knowledge databases.

The active component associated with each multimedia element is also afile containing a script in a platform independent language such as Javalanguage.

All files belonging to a multimedia element are organized in a commoncontainer. In other embodiments, they can be stored separately as files.

Additionally, the system contains an execution platform for the scriptsassociated with the multimedia data, for instance a JAVA executionenvironment. This JAVA execution environment is part of the intelligenceassociated with the storage medium 2.

The script associated with each multimedia element will execute thefollowing steps:

request of the current date and time to the multimedia asset managementsystem,

looking all time information available in supplementary files of themultimedia items,

comparing the current date and time with the creation date informationassociated with the multimedia data stored on the storage medium 2,

detection of temporal events such as:

-   -   the current date is equal to or is close to the item creation        date,    -   the current date equals to or is close to the anniversary of the        multimedia element creation date,    -   the anniversary of the creation date of the multimedia element        is within one week or in close-future,    -   the anniversary of the creation date of the multimedia element        is a particular anniversary (5.sup.th, 10.sup.th, 50.sup.th . .        .),

relating the detected temporal events with other supplementaryinformation of the multimedia element such as type of event, annotation

detection of anniversaries of weddings,

detection of the anniversary of holidays trips,

relating the detected temporal events with other usage-related,supplementary information associated with the multimedia data such as auser profile,

detecting anniversaries of multimedia data that were not created by thecurrent user,

detection of anniversaries of multimedia data that are very rarely orvery often accessed,

detecting the anniversary of items very often accessed by others thanthe current user,

generating remembering information from the detected temporal events andtheir relations:

presenting wedding photos for an anniversary of wedding,

announcing an audio clip by synthetic voice,

presenting a video clip from Christmas with indication of who is inthere.

In other embodiments, more particular actions can be carried outdepending on the user application.

The user interface 3 transforms the remembering information andintegrates them in the rendering information for a display 4. Thedisplay 4 consists of a screen and of loudspeakers to display graphicalinformation from the user interface 3 or to play sounds generated by theuser interface 3.

What is claimed is:
 1. A non-transitory storage medium for multimediaitem retrieval and adapted to store at least one multimedia itemassociated with at least one date and an active component adapted to:depending on time information received from a multimedia assetmanagement and the at least one date, send a remembering information toa user interface, the remembering information comprising a trigger of anaction that makes a user aware of the multimedia item.
 2. Thenon-transitory storage medium according to claim 1, wherein said atleast one date comprises a creation date of the at least one multimediaitem, said time information received from the multimedia assetmanagement is the date of the current day, and said active component isadapted to send said remembering information when the current day is ananniversary date of the creation date of the at least one multimediaitem.
 3. The non-transitory storage medium according to claim 1, whereinsaid action comprises playing or displaying the at least one multimediaitem.